Skip to main content

Use of clustering

The idea in this document to explore how we can leverage image clustering so that we can automate the process of video type annotation in the CCTV pipeline.

End goal​

  1. We want to automate the process of annotating different video types and their various distance regions.
  2. We are planning to use data/knowledge already available from the OCR annotation (which includes the extracted text and the bounding boxes)
  3. We want to use the extracted text and the bounding boxes to annotate the video.
info

We are assuming that the OCR annotation is already done and we have the extracted text and the bounding boxes for all the frames

Approaches​

  1. Extract the number of bounding boxes from each video
  2. For each bounding box, calculate its centroid and plot it as a point on a graph
  3. Try to cluster these centroids
  4. To improve the clustering, initialize the clustering algorithm with the mean centroids of all the frames

Questions that came up​

  1. How do we handle frames with different resolution when we stitch them and send them to Azure OCR?
  2. OCR has a tends to break up larger chunks of text into different bounding boxes. We are not sure, how will that affect the approach mentioned above
  3. We also need to consider the case when we have videos with different resolutions.